Gaze initiated interaction technique

ABSTRACT

A gaze initiated interaction technique for facilitating interaction between wearers of head-mountable devices. Methods are shown which allow users to interact with each other following an event of interlocking gaze. Methods are also shown which allows the operator of a personal point-of-interest beacon to control and limit interaction from the public thus ensuring a greater degree of privacy. Methods are also introduced which allow a mode of interaction to be determined by a remote service after mutual authorization of each interacting user. This disclosure also introduces a system of a head-mountable device which comprises a directional beacon transceiver system using beams in the infrared light spectrum enabling the detection of a gaze interlock event.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of the following U.S. patentapplication, which is incorporated by reference in its entirety:

-   U.S. Non-Provisional application Ser. No. U.S. Ser. No. 15/017,606,    filed Feb. 6, 2016;

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not applicable to this application.

BACKGROUND Field

Example embodiments in general relate to a gaze initiated interactiontechnique for inviting, initiating and performing interaction betweenwearers of head-mountable devices.

Prior Art

Any discussion of the related art throughout the specification should inno way be considered as an admission that such related art is widelyknown or forms part of common general knowledge in the field.

With the advent of wearable devices the concept of point-of-interestbeacons has received an increasing amount of attention in recent years,with the iBeacon technology becoming more and more prevalent. Currenttechnology is centered around beacons being assigned unique identifiers,such as UUIDs, and configuring these beacons to transmit their staticand permanent identifiers with a low range omnidirectional transmitter,such as Bluetooth LE. Whilst this technology is particularly suited toallow interfacing the point-of-interface beacons of large and inanimateobjects, such as buildings or storefronts, there are a number ofshortcomings which currently stand in the way of using point-of-interestbeacons to facilitate interpersonal communication.

SUMMARY

This disclosure provides, in part, a group of methods.

One method includes detecting the gaze of a person wearing ahead-mounted device and providing means for interacting with thatperson. Several means are known that allow detecting beacons associatedwith points-of-interest in an augmented reality environment. However,when a point-of-interest is associated with a person that person may notdesire being tracked or contacted by an unrestricted group of people.The method also provides a means to authenticate against apoint-of-interest beacon through mediation of a remote network, such asa social network.

Another method includes enabling the user of a head-mountable device toinvite interaction from other members of a shared network. The methodincludes providing a means for the beacon user to restrict visibilityand interaction with regards to other users. Therefore the beacon userremains anonymous to other users of similar head-mountable devices andonly becomes visible to a select group of members of a network. Themethod includes a means to configure the head-mountable device toproject the beam of the point-of-interest beacon in the direction of theuser's gaze.

Another method includes configuring a remote service to provideauthorization for interaction with a point-of-interest. The serverdetermines whether access should be granted based on inclusion orexclusion criteria set in user profiles. If access is granted, thefunction returns an interface to the point-of-interest.

This disclosure also provides, in part, a system of a head mounteddevice (HMD). The system includes a directional beacon transmitting aunique identifier. The unique identifier may be changed in regularintervals such as to avoid tracking and to provide a degree of privacyand anonymity. The system may implement the directionalpoint-of-interest beacon with an infrared light source. The system alsoincludes a means for receiving a directional point-of-interest beacon,which may be implemented by an infrared-spectrum camera pointing in thesame direction as the gaze of the user.

There has thus been outlined, rather broadly, some of the features ofthe gaze initiated interaction technique in order that the detaileddescription thereof may be better understood, and in order that thepresent contribution to the art may be better appreciated. There areadditional features of the gaze initiated interaction technique thatwill be described hereinafter and that will form the subject matter ofthe claims appended hereto. In this respect, before explaining at leastone embodiment of the gaze initiated interaction technique in detail, itis to be understood that the gaze initiated interaction technique is notlimited in its application to the details of construction or to thearrangements of the components set forth in the following description orillustrated in the drawings. The gaze initiated interaction technique iscapable of other embodiments and of being practiced and carried out invarious ways. Also, it is to be understood that the phraseology andterminology employed herein are for the purpose of the description andshould not be regarded as limiting.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will become more fully understood from the detaileddescription given herein below and the accompanying drawings, whereinlike elements are represented by like reference characters, which aregiven by way of illustration only and thus are not limitative of theexample embodiments herein.

FIG. 1A is a block diagram of an example embodiment.

FIG. 1B is a first perspective view of a gaze initiated interactiontechnique in accordance with an example embodiment.

FIG. 2 is a frontal view illustrating a technique for utilizinginterlocking gaze in respect of wearers of a head-mountable device.

FIG. 3 illustrates an example embodiment relating to establishingcontact with the user of a personal point-of-interest beacon.

FIG. 4 illustrates an example embodiment relating to running an app on ahead-mountable device relating to communications with the user of apersonal point-of-interest beacon.

FIG. 5 is a block diagram depicting examples of the type of interactionwhich can follow acquisition of a personal point-of-interest beacon.

FIG. 6 is a flow diagram relating to an example embodiment depictingcommunications between two personal point-of-interest beacon users.

FIG. 7 is a block diagram depicting the system architecture of anexample embodiment of a system utilizing a gaze initiated interactiontechnique.

DETAILED DESCRIPTION

Among the techniques known in the prior art are those point-of-interestbeacons which transmit static identifiers, commonly UUIDs (universallyunique identifiers). Even though a UUID can be anonymous at the point ofcreation, once it has been associated with a person it can be used totrack that person, which is considered a disadvantage in any scenarioinvolving a requirement of privacy. The present invention, in part,provides a technique which allows the user of head-mountable device topublicly broadcast a constantly changing identifier thus making itdifficult to track or identify the beacon user. At the same time thebroadcast identifier can serve the function to invite interaction from aselect group of member of a shared network, such as a shared socialnetwork. Importantly the novel technique does not necessarily restrictinteraction to other network users, i.e. social network friends,previously associated with the beacon user. The technique also providesmethods which allow beacon users to restrict access by setting profilefiltering criteria, such as by setting which profile parameters anothersocial network user must exhibit in order to be allowed to interfacewith the beacon user. Therefore the novel technique offers, in part, thepossibility that a social network, for example, can mediateauthentication and authorization between beacon users. Other thanauthentication and authorization, the technique can also serve to selectand serve the appropriate interface for interaction. There may be anumber of augmented-reality apps, which can provide such interface. Thedisclosed technique will allow a social network to select the mostappropriate interface based on an algorithm taking into account bothuser profiles and the context of a gaze interlock event. The techniquewhich is subject to this invention also proposes a system of ahead-mountable device comprising a beacon system, whereby both thetransmitter and the receiver beam are envisaged to be directional,whereas prior art embodiments of such beacons tend to beomnidirectional. The reason for seeking directional beams lies insolving the problem of how find a technological equivalent to the basichuman interaction of the gaze of two persons meeting. In a preferredembodiment the directional beacon transmitter is implemented as adirectional beam of light in the infrared spectrum, whereas thedirectional beacon receiver is implemented as an infrared-spectrumcamera facing the same direction as the gaze of the user of ahead-mountable device.

The block diagram of FIG. 1A illustrates the process flow of a beaconinteraction from the perspective of the user of a head-mounted devicelooking to interact with other users. The process starts 1000 with apersonal point-of-interest beacon being acquired 1001. At this point theuser has only an anonymous identifier to go by and requiresauthorization 1002 before the anonymous identifier can be used for anykind of purpose. The authorization is requested from a remote serverwhich can look up the anonymous identifier and match it with a profileon a network, such as a social network. As long as the authorizationprocess is successful, the remote server returns a handle to aninterface 1003. Once an interface to the point of interest is availablethe user is given the option to commence interaction with thepoint-of-interest 1004. If the user accepts that invitation forinteraction 1005, then an application is started which is capable ofusing the interface returned in step 1003. The application is executeduntil a disengagement event 1007 is received, which ends 1008 theinteraction.

The schematic diagram in FIG. 1B depicts the hardware required in thescenario of a preferred embodiment of the present invention. Bothpersons taking part in the interaction are wearing a head-mountabledevice 140. The head-mountable device comprises both a beacontransmitter 110 and a beacon receiver 120. In a preferred embodimentboth the beacon transmitter and the beacon receiver are configured tooperate with light of the infrared spectrum. Both the transmitting andreceiving beams are configured to coincide with the gaze of the user130. Through the medium of the beacon beams two functions are beingimplemented. One such function is the exchange of the describedanonymous unique identifiers 180. The other function is to track thepoint-of-interest beacon 175 in order to be able to project thecorresponding beacon user 170 as an augmented reality target. In thepreferred embodiment both interacting users 170, 175 use a system whichcombines a head-mountable device with a smartphone-type handset 150. Itis envisaged that all distributed functions with the exception of saidbeacon tracking and UUID exchange are processed by making use of cloudservices 160 which are in turn accessed by functionality provided by therespective smartphone-type handsets 150.

FIG. 2 illustrates the gaze initiated interaction technique in thecontext of an augmented-reality setting whereby the figure depicts thefirst step of the technique comprising scanning the environment forother personal point-of-interest beacons in the field of view. The user250 of a head-mountable device gazes in the direction of a crowd ofpeople 235 whereby in the crowd there is a person 230 who operates asimilar head-mountable device comprising a point-of-interest beacon 220.The beacon 220 disseminates a beam 210 corresponding to the gaze of theuser 230. The first user 250 operates a similar beam transceivercorresponding to her gaze 240. Once an intersection of gaze-relatedbeams 260 takes place, unique identifiers can be exchanged. In thedepicted embodiment, however, it is sufficient if one of the users picksup the unique identifier of the beacon of the other user.

FIG. 3 illustrates the gaze initiated interaction technique from thepoint of view of a user 250 at a point in time when authorization hasbeen successfully performed on a plurality of social networks. The frame305 depicts the boundary of the augmented-reality scene depicted. Theframe contents simulate what the user 250 may see. Frame 350 denotes amenu panel superimposed on the augmented reality scenery. Frame 340denotes an individual selectable menu item. The scenery comprises acrowd of people which are not enhanced by augmented reality. There isone individual 310 in the crowd who is enhanced in the augmented realityenvironment. The enhancement consists of providing a means of tracking320 the individual in the field of view. An icon 330 hovers above thetracked individual 310. The said icon denotes that the trackedindividual 310 offers an invitation for interaction to the other user250. At this point of the interaction process no interface forinteraction is available yet. In the example the user 250 is notifiedthat authorization has been granted for interaction with thepoint-of-interest by altogether three social networks 361, 362, 363. Thesaid user is prompted to select over which one of the social networksoffered as selection the proposed interaction is to take place. Theselection of network is important for the reason that the interfaceoffered for interaction will normally be different for each network. Iffor example, the point-of-interest grants authorization both forinteraction over a social network designed for dating and a socialnetwork designed for professional networking, then it is to be expectedthat the corresponding interfaces are substantially different.

FIG. 4 depicts the use case started in FIG. 3 at the next phase ofinteraction. At this stage a preferred social network has been selected,the corresponding interface has been retrieved and an application hasbeen launched which makes use of the said interface. Whilst theapplication is active a suitable icon 400 is projected above the trackeduser 310 denoting that the currently active application interacts withsaid tracked user. Menu card 410 denotes an arbitrary interactionoption, in this particular example the option to send a text message tothe tracked user 310.

FIG. 5 is a block diagram providing more detail on preferred embodimentsof the type of interaction between users of the gaze initiatedinteraction technique. Once an interface is obtained enabling contact toa personal point-of-interest beacon an application 500 is launched. Themost basic application of that type would be one in which the seconduser is added as a friend 501 on the social network which has mediatedthe described authorization. Another possible application is theexchange of business cards, such as VCARDs 502. Sending a short textmessage 503 to the second user is another possible example application.In a similar way two way voice communications 504 may be initiated. Theapplication may also restrict interaction simply to tracking the seconduser as an augmented-reality target 505. This use case would beparticularly interesting in keeping track for example of a tour guide ina very crowded place. Another application would be to simply retrieveand browse profile attributes of the second user 506. In a gamingscenario it would be of particular interest to utilize the describedgaze initiated interaction technique to initiate an application whichallows to introduce a previously not acquainted member of the publicinto a running instance of an augmented reality game.

FIG. 6 describes an end-to-end use case of a gaze-initiated interactionbetween a first user 250 and a second user 310 in an augmented-realityenvironment. The second user 310 is operating a personalpoint-of-interest beacon transmitting an anonymous temporary uniqueidentifier which is only known to one social network. Both the first andthe second user use head-mountable devices 695 in combination withsmartphone-type handheld mobile devices 150. The interaction starts withthe first user 250 detecting the point-of-interest beacon 601 of thesecond user 310. Upon detection of the point-of-interest beacon theunique identifier of the beacon is retrieved. It has to be noted that apoint-of-interest beacon may transmit more than one identifier. If thatis the case, the following lookup process is repeated for each of theidentifiers. In the described use case, however, it is assumed that onlyone unique identifier is being transmitted. For the next step thehandset 150 of the first user 250 establishes a connection with theInternet in order to access a plurality of cloud services. From thepoint of view of the first user the input 620 for the lookup process isthe unique ID 698 of the beacon relating to the second user, whereas theoutput 625 is a service interface to the second user. The steps requiredto retrieve the service interface based on the lookup of the uniqueidentifier are as follows: at first the identifier is submitted 630 to alookup service of a first social network 670, which is a social networkon which the first user 250 has an account and a profile; if the uniqueidentifier 698 is registered on that social network an algorithm 685comprising a matching logic is executed, in such a way that thealgorithm takes into account inclusion and exclusion criteria associatedwith both the profile of the first user 250 and the second user 310; theoutcome of the matching process determines whether authorization isgranted to the interaction request; in the present use case lookupprocess 630 returns a negative authorization result 635; the next stepscomprise iterating through a plurality of social networks to which thefirst user 250 has subscribed; for each social network the lookupprocess is repeated; in the specific use case lookup 640 is successfulsince the matching algorithm 690 yields positive authorization and as aresult the social network 675 grants authorization for interaction 645;once authorization has been obtained it is necessary to obtain anapplication for interaction, for that purpose the social network 675 isqueried for the identifier of a suitable application; subsequently anidentifier for the respective application is obtained 655; in order tobe able to run the application instance data has to be obtained 657,also from social network 675; before the application can be launched itis necessary to ensure that the application itself is available on themobile devices of the first user 250; in the present use case theapplication has not yet been installed and has to be requested 660 anddownloaded from an online app repository 680. Once authorization hasbeen granted and the application is running and has been provided withinstance data, then the user 250 is notified that augmented-realityinteraction with user 310 is possible through application instance 606.Both users 250 310 are queried to agree to a virtual handshake whichinitiates interaction through the app 606, if affirmed. For as long asthe application 606 is active both users track each other in theaugmented-reality environment 615.

FIG. 7 depicts the architecture of an example embodiment of a systemimplementing aspects of the novel gaze initiated interaction technique.The system comprises two mobile devices, namely a head-mountable device740 and a smartphone-type mobile device 745. Both devices 740 745 areused in conjunction with each other and are thus treated as a singleunit for the purposes of the system diagram. The system comprises one ormore processing units 705, a transceiver 710 for mobile communicationsvia the smartphone-type device 745 and a plurality of communicationinterfaces 715. The device comprises a data storage component 739, whichin turn comprises stored program logic 730 and stored program data 735.The system also comprises a plurality of user input and output devices270. The plurality of input and output devices comprises a beaconprocessor 759. The beacon processor comprises a directional beacontransmitter 751, a directional beacon receiver 753 and a beacon tracker753. Other user input and output devices in the preferred embodiment area front-facing camera 761, a display 762, a speaker 763, a microphone764 and a touchpad 765.

Unless otherwise defined, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. Although methods and materialssimilar to or equivalent to those described herein can be used in thepractice or testing of the wink gesture based control system, suitablemethods and materials are described above. All publications, patentapplications, patents, and other references mentioned herein areincorporated by reference in their entirety to the extent allowed byapplicable law and regulations. The gaze initiated interaction techniquemay be embodied in other specific forms without departing from thespirit or essential attributes thereof, and it is therefore desired thatthe present embodiment be considered in all respects as illustrative andnot restrictive. Any headings utilized within the description are forconvenience only and have no legal or limiting effect.

What is claimed is:
 1. A device for processing gaze-interlock events,the device comprising: a gaze-interlock event detection circuit fordetecting when the gazes of two persons are intersecting and forreceiving an anonymous identifier as part of said detection operation;and a gaze-interlock event processing unit, configured in such a waythat, conditional on the detection of a gaze-interlock event: saiddevice queries a social network for information relating to a user ofsaid social network, wherein said query requires said identifier as akey, and wherein said device queries a plurality of social networks withsaid identifier and wherein said device causes a person involved in agaze-interlock event to be tagged in augmented reality and wherein saidtagging is conditional on a response to said querying.
 2. The device ofclaim 1, wherein said event detection circuit comprises a beaconreceiver.
 3. The device of claim 2, wherein said beacon receivercomprises a sensor for infrared-spectrum light.
 4. The device of claim2, wherein said event detection circuit is configured to detect onlyinside a restricted detection space.
 5. The device of claim 4, whereinsaid restricted detection space substantially corresponds to a field ofview of a human being.
 6. The device of claim 4, wherein said device isconfigured to be body-worn and wherein said restricted detection spaceis configured to be in front of the body of a wearer of said device. 7.The device of claim 4, wherein said device is configured to be head-wornand wherein said restricted detection space is configured to be centeredaround a line of sight of a wearer of said device.
 8. The device ofclaim 1 wherein said device is logged into an account on said socialnetwork and wherein said account is associated with a plurality ofinclusion or exclusion criteria with respect to being contactable byusers of said social network following a gaze-interlock event.
 9. Anon-transitory computer readable storage medium having data storedtherein representing software executable by a computer, the softwareincluding instructions to process gaze interlock events, the storagemedium comprising: instructions for receiving event notifications from adetector for gaze-interlock events wherein said events are triggeredwhen the gaze of a person is being directed in the direction of thedetector; and contingent on the detection a gaze-interlock event; andinstructions for tagging said person in augmented reality wherein saidtagging is conditional on the responses received by querying a pluralityof social networks, wherein said queries are parametrized with ananonymous inbound identifier received from said detector.
 10. Thecomputer readable storage medium of claim 9, wherein said detectorcomprises an infrared-spectrum camera.
 11. The computer readable storagemedium of claim 9, wherein said detector is configured in such a waythat a field of detection is directional.
 12. The computer readablestorage medium of claim 9, wherein said detector is configured in such away that thea field of detection is modelled on a field of view of ahuman person.
 13. The computer readable storage medium of claim 9further comprising instructions for causing an infrared transceivercircuit to exchange a set of anonymous identifiers with an externalsystem and wherein said exchange is in response to a gaze-interlockevent.
 14. The computer readable storage medium of claim 13 furthercomprising instructions for replacing, in regular intervals, theoutbound anonymous identifier.
 15. The computer readable storage mediumof claim 14 further comprising instructions for requesting from a socialnetwork an authorization for the opening of a communication channel andwherein said authorization is conditional on querying said socialnetwork with said inbound identifier.
 16. A method for reacting togaze-interlock events, the method comprising: detecting a gaze-interlockevent and in response to said event being detected: identifying a personby querying a plurality of social network with an anonymous identifier,wherein said identifier is picked up as part of the detecting step andwherein said social network is capable of linking said identifier to auser profile of said person, and wherein identifying said personcomprises tagging said person in augmented reality conditional on theresponse of said querying.